Lecture 2 - The Research Design Process
Lots of additional readings included in the syllabus 😊
https://forms.gle/xUL39k7ngY2kXYJC7
Source: University of Cambridge
As researchers, we are interested in research questions about how the world works
There are a number of different types of questions that we may want to answer. In academia, they are often divided into two broad categories:
Then we can move on to questions about why? \(\rightarrow\) i.e., knowing the effect of a cause is necessary before moving on to understanding the causes of an effect.
(Next sessions: more on about what we mean by causality and how experiments give us leverage to make causal claims.)
What is the phenomenon we want to explain?
Does the cause we theorise lead to observing changes in \(Y\)?
What is the theory of change?
We are ultimately interested in how two theoretical concepts are related, measured by observed variables \(T\) (our treatment) and \(Y\) (our outcomes)
There is no such thing as “just doing an experiment” 🧐
All research design involves theory, whether implicit or explicit
Our questions are value laden: For example, social scientists studied marijuana use in the 1950s as a form of “deviance”, the questions focused on “why are people making such bad decisions?” or “how can policy makers prevent marijuana use?”
Why do the research? We might want to change how scientists explain the world and/or change the policy decisions in (a) one place and time and/or (b) in other places and times
Research focused on learning the causal effect of \(T\) on \(Y\) requires a model of the world: how might intervention \(T\) might have an effect on some outcome \(Y\), and why, and how large might be the effect. It helps us think about how a different intervention or targeting different recipients might lead to different results.
Our theories and models are important not just for generating hypotheses, but for informing design and strategies for inference
Designing research will often clarify where we are less certain about our theories. Our theories will point to problems with our design. And questions arising from the process of design may indicate a need for more work on explanation and mechanism
What is the outcome of interest (\(Y\))?
What is the cause of interest (\(T\))?
What can be a theory that yields to this experimental design?
What can be the main hypothesis?
How can we measure our outcomes?
Can we directly manipulate \(T\)? (underlying treatment concept of interest)
How does T relate to \(T\)?
Did everyone receive \(T\)?
Source: World Health Organization, 2003
Now think of yourselves as the researchers
In pairs or groups of three:
We often cannot directly observe the true value of the outcome concept for most of the outcomes we are interested in
Examples:
Moreover, the underlying outcome concept may be even under debate (e.g., democracy)
If our indicators don’t measure the underlying concept that we’re interested in, then we may not be able to learn very much, even if we have an otherwise very sound experiment